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Abstract 

Quality Indicators (QIs) are measures of health care quality that make use of readily available hospital inpatient 
administrative data. Assessment quality of care can be performed on different levels: national, regional, on a 
hospital basis or on an individual basis. It can be a mandatory or voluntary system. In all cases development of an 
adequate database for data extraction, and feedback of the findings is of paramount importance. In the present 
paper we performed a Medline search on "QIs and breast cancer" and "benchmarking and breast cancer care", 
and we have added some data from personal experience. The current data clearly show that the use of QIs for 
breast cancer care, regular internal and external audit of performance of breast units, and benchmarking are 
effective to improve quality of care. Adherence to guidelines improves markedly (particularly regarding adjuvant 
treatment) and there are data emerging showing that this results in a better outcome. As quality assurance benefits 
patients, it will be a challenge for the medical and hospital community to develop affordable quality control sys- 
tems, which are not leading to excessive workload. 
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Introduction 

High-quality services are essential to optimize treat- 
ment results of women with breast cancer. They can 
be achieved by accurate training, specialization, 
volume levels and a multidisciplinary approach, in- 
volving many different subspecialists, nursing staff 
and supporting staff members (Perry et al., 2008). 
Due to the increasing complexity of the breast teams 
it is of paramount importance to develop structured 
care in order to avoid a chaotic and non-consistent 
management of patients and ameliorate adherence 
to guidelines. 

It has been reported that in Europe there are wide 
differences in treatment offered to patients with 
breast cancer in terms of mastectomy and radiother- 
apy rates and the use of adjuvant chemotherapy and 
hormone therapy, which result in considerable sur- 
vival differences (Del Turco et al., 2010). A recent 
Italian survey showed adherence to the breast can- 



cer guidelines of the Italian Association for Medical 
Oncology was only seen in 71% of 355 cases exam- 
ined in 35 representative Italian oncological units 
(Bami et al., 201 1). The Florida Initiative for Qual- 
ity Cancer Care (a consortium of three academic 
and eight community hospitals in Florida) is a 
physician and practice-based quality improvement 
project that was conceived to study the barriers of 
delivering high quality cancer care in Florida (Gray 
et al., 2011). Of the 34 quality indicators (QIs) for 
breast cancer care they evaluated, seven for medical 
oncology and four for surgical oncology fell below 
the 85% level of adherence. A national process sur- 
vey in the USA providing benchmark data showed 
that a crucial element in the treatment of women 
with breast cancer, namely determination of the 
estrogen- and progesterone receptor status was per- 
formed in only 83% and 81% of the patients (Owen 
et al., 2009). Adherence to quality measures was 
less than 85% for 18 of the 36 defined breast cancer 
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quality measures (Malin et al 2006) . These observa- 
tions highlight a gap between optimal and actual 
care, that is, between what evidence has identified 
as recommended care and what patients actually 
receive (Asch et al., 2006). There is a world-wide 
need for tools to improve quality of care in the daily 
clinical practice. 

QIs are measures of health care quality that make 
use of readily available hospital inpatient adminis- 
trative data. They can be used to highlight potential 
quality concerns, identify areas that need further 
study and investigation, and track changes over 
time. We have recently shown that the use of QIs 
for breast cancer care, combined with regular inter- 
nal and external audits of performance, are effective 
to improve adherence to guidelines in a breast can- 
cer unit (van Dam et al., 2013). Assessing quality of 
care can be performed on different levels: national, 
regional, on a hospital basis or on an individual 
basis. It can be a mandatory or voluntary system. 
In all cases development of an adequate database 
for data extraction is of major importance. In the 
present paper we performed a Medline search on 
"QIs and breast cancer" and "benchmarking and 
breast cancer care", and we have added some data 
from personal experience. 

Assessing process and outcome parameters by 
means of population based registries 

Various groups have called for a national system to 
monitor the quality of breast cancer care extracting 
data from existing cancer registries for the entire 
population. This allows scrutinizing breast cancer 
care in an entire region or country without excep- 
tion of certain breast clinics. By analyzing perfor- 
mance on a national basis, one can identify more 
global and system-level elements that can be 
addressed to achieve more widespread quality 
improvement efforts (Chen et al., 201 1). 

In order to do this type of analysis it is mandatory 
that the registries contain a minimum number of 
parameters which can be used as QIs, the data 
should be reliable and the data sets complete for the 
vast majority of patients. Malin et al. (2002) studied 
the validity of such information in the California 
Cancer Registry by comparing the individual re- 
cords of 304 women with a new diagnosis of breast 
cancer with the registry data. The accuracy of regis- 
try data was higher for hospital based services (eg. 
sensitivity -95% for mastectomy, 95% for lumpec- 
tomy, 96% for lymph node dissection) than for am- 
bulatory services. They concluded that registries 
could provide the infrastructure for collecting data 
on the quality of cancer care, although the cancer 
registry data may not be valid for all care settings. 



Caldarella et al. (2012) confirmed that it is feasi- 
ble to evaluate quality of cancer care using cancer 
registry population based data and major computer- 
ized information systems. They tested a set of breast 
cancer care QIs on the Tuscan Cancer Registry 
which collects tumor cases diagnosed in all residents 
in Tuscany. The estimation of the selected indicators 
confirmed a good homogeneity among areas, and 
globally a good intraregional performance. 

In a similar survey McCarthy et al. (2008) col- 
lected data sets relating to treatment of four com- 
mon cancers (breast, colorectal, lung and prostate) 
from the data bases of 188 National Health Service 
hospital trusts in the UK. Data sets were estimated 
mostly 80% complete. Process indicators were eval- 
uated and patient outcome measures included the 
National Survey of Patients Satisfaction for Cancer, 
and cancer survival drown from cancer registration. 
They showed that while this datamining system of 
evaluation of quality of care was not yet used in 
practice, it is possible to implement this in perfor- 
mance assessment of cancer networks. 

Recently a French group (Ferrua et al., 2012) has 
identified 8 QIs to evaluate the timelines of breast 
care. They feel that 3 of them are ready for nation- 
wide implementation in France (time to surgery, 
time to postoperative multidisciplinary team meet- 
ing, conformity to decision of multidisciplinary 
team meeting). 

The team of the Belgian Health Care Knowledge 
Centre developed a set of indicators to monitor the 
quality of breast cancer care in Belgium (Stordeur et 
al., 2012). QIs were identified from a systematic 
literature search and the 2010 Belgian evidence 
based clinical practice guideline. The selection pro- 
cess involved an expert panel evaluating reliability, 
relevance, interpretability and actionability of each 
indicator. The QIs were tested using the Belgian 
Cancer Registry data linked with claims data for all 
women registered with breast cancer in Belgium be- 
tween 2001 and 2006 (N = 50.039). This selection 
process led to a final set of 32 indicators. Of these, 
12 were measurable using the available data, while 
one indicator was measurable using proxy 
information. The authors showed that by data 
mining, linking the National Cancer Registry to 
claim data, these 13 indicators could be used as a 
multidisciplinary set of QIs for breast cancer. In a 
second study on the same data set they selected the 
women treated with invasive breast cancer between 
2004 and 2006 (Vrijens et al., 2012). A total of 
25.178 women had their treatment in 1 18 hospitals. 
These were classified according to their annual 
volume of treated patients: < 50 (very low), 50-99 
(low), 100-149 (medium) and > 150 (high). Six of 
the eleven process indicators showed higher rates in 
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high-volume hospitals: discussion of the case in a 
multidisciplinary team meeting, cytological and/or 
histological assessment before surgery, use of neo- 
adjuvant chemotherapy, breast conserving surgery 
rate, adjuvant radiotherapy after breast conserving 
surgery, and follow-up mammography. Higher vol- 
ume was also associated with improved survival. 
The 5-year observed survival rates respectively 
were 74.9%, 78.8%, 79.8% and 83.9% for patients 
treated in very-low, low-, medium- and high- vol- 
ume hospitals. After case-mix adjustment, patients 
treated in very-low or low-volume hospitals had a 
hazard ratio for death of 1.26 (95% CI: 1.12-1.42) 
and 1.15 (95% CI 1 .01- 1 .30) respectively compared 
to high volume hospitals. The authors concluded 
that the survival benefits reported in high volume 
hospitals suggest a better application of recom- 
mended processes of care, justifying the centraliza- 
tion of breast cancer care in such hospitals. It is 
clear that this type of national audits may lead to 
adaptation of policies. A result of the above report 
was the political decision to implement the prospec- 
tive evaluation of 17 QIs in the recognized breast 
clinics in Belgium, which up to now should treat at 
least 100 new patients a year. These QIs will be 
centrally evaluated on a continuous basis by the 
government and breast clinics should perform 
according to a minimum standard (based on these 
17 QIs) in order to keep their certification. 

Quality assurance through voluntary bench- 
marking and certification 

During the last decade prospective voluntary bench- 
marking procedures of breast units have been estab- 
lished. Breast clinics have to keep a prospective da- 
tabase of all breast cancer patients they treat and 
this is at least annually audited by an independent 
external organization. This can be done electroni- 
cally on the data in the database or by a regular in 
vivo site visit of the unit. Breast clinics meeting a 
minimum standard of quality of breast cancer care 
get a certification for a certain amount of years. This 
accreditation has to be renewed on a regular basis. 
A major advantage of this system is that the quality 
of the data entered in the database is superior to data 
which are retrieved by dataprocessing of national 
registries. The down side is that only units who are 
willing to comply to the certification process are 
looked at. 

In collaboration with the Deutsches Onkologie 
Centrum a prospective interventional multicentre 
study was started in 2003 in Germany in order to 
establish a supraregional collaborative network of 
breast centers to provide proof of concept for cen- 
tralized collection and independent analysis of rel- 



evant quality assurance data, and to define suitable 
QIs for benchmarking the quality of breast cancer 
care (Brucker et al., 2008). Data collection depend- 
ed on voluntary participation of breast centers in a 
commercial benchmarking procedure conducted by 
an independent external institute. Patient data were 
anonymised, encrypted and submitted for analysis 
twice a year. Nine guideline-based clinical parame- 
ters designed to attainment of predefined quality 
targets were defined as rate-based QIs. The DKG/ 
DOS dual certification process in its present form 
was established in July 2003. Essentially it com- 
bines compliance with the Requirements of Breast 
Centres (Fachliche Anforderungen fur Brustzentren 
developed by DOS , DKG) and implementation and 
maintenance of a certified quality maintenance sys- 
tem. During the period 2003 to 2010 the number of 
certified breast centers increased form 59 to 210. In 
2010 about 90% of the new breast cancer cases in 
Germany, currently estimated at about 57970 per 
year, were diagnosed and treated in a certified breast 
centre (Wallwiener et al., 2012). The initial set of 
nine QIs had increased to 18 QIs as surrogate indi- 
cators of long-term outcome quality. The 2003- 
20 10 period saw marked increases for the following 
QIs: preoperative histological diagnosis from 58 to 
96%, guideline-concordant endocrine therapy in 
hormone receptor-positive patients from 27 to 97%; 
guideline accordant adjuvant and neoadjuvant che- 
motherapy (no age limit) from 32 to 78%; radio- 
therapy after breast conserving surgery from 20 to 
87%; and radiotherapy after mastectomy from 8 to 
74%. There is no doubt that the German voluntary 
program for external benchmarking has produced 
remarkable results. It is highly likely that the 
improved quality of breast cancer care will result 
in better outcomes. The collection of longitudinal 
follow-up data on the entire population of patients 
treated in certified breast centers is ongoing to look 
at the effects of the treatment related interventions 
on survival of patients. Beckmann et al. (2011) 
looked at a sample of 3940 of these patients treated 
for invasive breast cancer in Middle Franconia 
(Germany). Patients undergoing treatment in certi- 
fied breast centers were younger, had lower disease 
stages and lower grading. Independent of the effects 
of these variables, patients treated in a certified 
breast centers had a better overall survival in the 
adjusted Cox model (hazard ratio 0.70; 95% CI: 
0.52-0.93. 

The European Society of Breast Cancer Special- 
ists - EUSOMA - has started a voluntary certifica- 
tion process to assess the clinical performance in 
dedicated European units (Perry et al., 2008; Greco 
et al., 2006; Del Turco et al., 2010). So far, 32 breast 
units in Europe have been recognized to comply 
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with the requirements requested by EUSOMA and 
other European Union guidelines on the basis of 
information collected by a questionnaire and by a 
site visit carried out by an independent team of 
breast cancer experts. A set of QIs was defined by 
experts from different disciplines based on a litera- 
ture review. For each of them they reported the def- 
inition, minimum and target standard, motivation 
for selection and level of evidence. Overall 17 main 
QIs have been identified, respectively 7 on diagno- 
sis, 4 on surgery and loco-regional treatment, 2 on 
systemic treatment and 4 on staging, counseling, 
follow-up and rehabilitation (Del Turco et al., 
2010). EUSOMA has selected 10 basic indicators 
(Table 2) to be used for certification purposes. These 
clearly defined quality parameters, continuous in- 
ternal audit and external social control by means of 
a site visit are of paramount importance to optimize 
adherence to evidence based guidelines and treat- 
ment results. Specialized breast units should further 
comply to the following requirements: Breast sur- 
gery should be performed by a specialist in breast 
surgery. There should be specialized radiothera- 
pists, medical oncologists, pathologists, radiologists 
and breast nurses to take care of breast cancer pa- 
tients. The unit should treat at least 150 new cases 
with breast cancer a year should have specialized 
clinics, genetics, psychological and social support 
available (Perry et al., 2008). Looking at the perfor- 
mance of individual units (see below), there is no 
doubt that this system improves the quality of breast 
cancer care. Currently an analysis is ongoing on the 
data of the entire population of women treated in 
EUSOMA certified centers, consisting of more then 
64000 patients, to assess the impact of this system 
of audit and certification on the performance of 
treatment of women with breast cancer. 

In the United States the National Accreditation 
Program for Breast Centers (NAPBC) is in the pro- 
cess of being developed. Accreditation is granted 
only to those centers that voluntarily committed to 
provide the best care in the diagnosis and treatment 
of breast cancer and are able to comply with NAPBC 
standards. Each center must undergo a rigorous 
evaluation and review of its performance and com- 
pliance with NAPBC standards. The most efficient 
methods of data collection are currently being de- 
fined (Winchester, 2011) 

Audit of breast cancer care in individual hospi- 
tals 

From an institutional standpoint analysis of QIs 
identifies areas of good and lower quality in the 
continuum of breast cancer care. Indicators with 
< 85% performance indentify specific clinical 



processes in most need for quality improvement. 
Hospitals in the Midwestern part of the Netherlands 
carried out a clinical audit to monitor the quality of 
breast cancer care during the years 2002-2008 
(Verbeek et al., 2011). Nine QIs were evaluated 
over the years. In 2004 and 2005 the hospitals also 
carried out an intervention project aimed at amelio- 
ration in quality over time. At the end of the project 
all nine indicators showed significant improvement 
compared to the start of the project. Discussion of 
treatment before and after surgery took place more 
often (respectively 83 % versus 53% and 96 versus 
70%. The National Guideline for maximum waiting 
times was met more often for the outpatient clinic 
(74% versus 61%), time to diagnosis (92% versus 
82%), and surgical treatment (52% versus 34%). 
More sentinel lymph node procedures were per- 
formed successfully (92% versus 69%), and for 
more patients more than 10 lymph nodes were eval- 
uated in case of axillary lymph node dissection 
(85% versus 58%). More patients had definitive 
surgical treatment consisting of one surgical inter- 
vention (87% versus 75%). 

Rizzo et al. (201 1) reported the compliance with 
the three National Quality Forum guidelines before 
(2005-2006) and after (2008) implementations in 

2007 in a metropolitan public hospital in Atlanta. 
Patients receiving radiation therapy increased from 
76% tot 96%. Patients receiving or considered for 
adjuvant chemotherapy or hormonal therapy in- 
creased from 74% to 94% and from 84% to 90% 
respectively (Iyengar et al., 2010). 

Weber et al. (2012) assessed the adherence to 
Breast Cancer Care Quality Indicators (BCCQI) 
before and after the introduction of a patient naviga- 
tion program helping to guide patients through the 
complex multidisciplinary cancer system, in a retro- 
spective cohort of 134 consecutive patients treated 
between 1" January 2006 and 31"' December 2006 
and 234 consecutive patients between 1" January 

2008 and 31"' December 2009. Overall, compliance 
with BCCQI improved from 74.1 to 95.5% 
(p < 0.0001). The impact of disease-free and overall 
survival remains to be determined. Similar observa- 
tions were reported by Chen et al. (201 1) assessing 
QIs in a cohort of 105 patients with newly diagnosed 
stage I-III breast cancer patients in a Los Angeles 
public hospital. 

Onerheim et al. (2008) showed that surveillance 
of the quality of surgical pathology reports of 
patients undergoing segmental resections of breast 
cancer was associated with significant improvement 
in the quality of the reports. Fifty one Quebec 
hospitals participated in a voluntary project in 1999 
and 50 in 2003. Overall, conformity improved from 
85% in 1999 for the first evaluation, to 92.5% in 
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Table 1. — Process indicators in thie period 2002-2011 in 
Hospital . Data were collected in the first semester of each 


the clinical pathway "operable breast cancer" 
year. 


of the Sint Augustinus 


Indicator 




2002 


2011 


P 


level 


Average hospital stay/patient (days) 




7.0 


4.1 


P 


<0.01 


Breast conserving surgery (%) 




43% 


58% 


n 


<0.01 


Preoperative guide wire (%) 




14% 


27% 


p 


<0.01 


Sentinel node biopsy (%) 




0% 


49% 


p 


< 0.001 


Preoperative staging tests missing (%) 




53% 


8% 


p 


<0.01 


Proportion second surgery (%) 




25% 


10% 


p 


< 0.005 


Clear margins at last surgery (%) 




95% 


99.5% 


p 


= 0.016 



2003 for the second evaluation (p < 0.001). Six of 
the 7 indicators showed an improvement between 
the first and second evaluations. Conformity was 
weakest for recording the distance of the resection 
margins (68%) and presence of lymphovascular in- 
vasion (61%) in 1999. 

We recently performed an analysis of evolution 
of the EUSOMA quality indicators in the breast unit 
of Sint Augustinus Hospital for the period 2002- 

2010 (van Dam et al., 2013). Multiple QIs were col- 
lected prospectively, and feed back and discussion 
of data with the breast team was organized at least 
annually. If necessary corrections in policy were 
made based on the internal audit data. Process indi- 
cators are given in Table 1 for 2002 and 20 1 1 . Evo- 
lution of the major mandatory quality indicators 
defined by EUSOMA (www.eusomadb.org/indica- 
tors.htm) is shown in Table 2. It can be seen that 
most of these indicators are significantly better in 

2011 compared to 2003. EUSOMA criteria were 
met for all indicators in 2011. Four years progres- 
sion free survival was significantly (p < 0.05) better 
in the cohort of patients treated in 2006-2008 com- 
pared to 2004-2005 and 2002-2003. Similar results 
were obtained if our data were stratified for Tl tu- 
mors only and T2-T4 tumors (van Dam et al., 2013). 
Although our survival results can partly be explained 
by an evolution in the case mix, with considerably 
more patients with small tumors and negative lymph 
nodes in more recent years, better adherence to 
guidelines is likely to be beneficial for the outcome 
of the patients. In 2011 more the 97% of patients 
had state of the art adjuvant radiotherapeutic, anti- 
hormonal or cytostatic treatment when indicated 
according to Sankt Gallen guidelines compared to 
98%, 85% and 72% respectively in 2003. 

A recent Taiwanese study shows that when breast 
cancer patients are diagnosed and treated in com- 
plete accordance with widely accepted standards of 
care, they survive longer and have better outcomes 



(Cheng et al., 2009). This prospective study fol- 
lowed 1378 newly diagnosed breast cancer patients 
from 1995-2001 in a single cancer hospital, tracking 
10 indicators of care quality and a assessing the pro- 
gression of disease up to June 2007. Adherence to 
all 10 QIS by 100% of patients was associated with 
better overall (HR 0.46, 95% confidence interval 
0.33-0.63) and progression-free survival (HR 0.51; 
95% confidence interval: 0.39-0.67). Adherence to 
either the four treatment indicators, or the six diag- 
nostic indicators by 100% of patients was also as- 
sociated with a significant improvement of survival. 

Audit of breast cancer care on an individual basis 

Nowadays it is perfectively feasible to give indi- 
vidual physicians feedback on their treatment re- 
sults. Most national and hospital databases collect 
data on the specialists who have participated in the 
treatment of breast cancer patients. Although one 
has to be very careful to audit individual perfor- 
mance of hospital physicians, as differences in case 
mix may bias the conclusions, it may be useful to do 
this type of analysis to identify possible points of 
amelioration in breast care. If feedback can be given 
on these findings in discrete and a non-hostile way 
it can help doctors to adapt certain habits and to re- 
flect more on their clinical activities. An internal 
audit in the Sint Augustinus hospital in 2007 showed 
for example that patients treated by the core breast 
team compared to patients treated by other collabo- 
rating physicians had a preoperative core biopsy 
rate of 70% versus 62%, a mastectomy rate of 25% 
vs 42%, free margins of 100% versus 97%, use of 
sentinel node biopsy in T1-T2 of 82% versus 35%. 
Feedback of these findings and discussion of indi- 
vidual cases in the multidisciplinary meetings im- 
proved performance of both groups. For example, 
preoperative histological diagnosis was nearly 90% 
for all patients treated in Sint Augustinus in 20 1 1 . 
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Table 2. — Evolution of quality indicators as formulated by EUSOMA prospectively evaluated between 2003 and 20 1 1 (data 
for first semester breast clinic Sint Augustinus Hospital, Antwerp). 


Outcome measure 


2003 


2011 


p-level 


Minimum 
Standard 


Positive preoperative cyto/nistological diagnosis 


59.7% 


88.3% 


0.0001 


80% 


Operated invasive carcinoma for which histological type, 
Grading, ER/PR status, stage and size were recorded 


94.1% 


96.4% 


NS 


90% 


Operated non invasive carcinoma for which size. 
Histological type and grading are recorded 


72% 


100% 


NS 


90% 


More then 9 lymph nodes removed when axillary 
Lymph node dissection performed (sampling excluded) 


85.6% 


86.4% 


NS 


85% 


Postoperative radiotherapy in non-metastatic invasive 
Carcinoma treated with breast conserving surgery 


98% 


97.3% 


NS 


90% 


Breast conserving surgery in invasive carcinoma with total size up 
to 30 mm (including DCIS component) 


62% 


88% 


0.001 


70% 


Breast conserving treatment for in situ carcinoma up to 20 mm 
diameter 


43.8% 


88% 


0.0016 


70% 


Ductal carcinoma in situ without axillary dissection 


85.7% 


97.1% 


NS 


80% 


Hormonotherapy in endocrine sensitive invasive carcinoma 


84.8% 


93.4% 


0.0002 


80% 


Adjuvant chemotherapy in ER negative (pTlc+ or N+) 
Invasive carcinoma 


72% 


97.1% 


0.0280 


80% 


N: number; ER: estrogen receptor; PR: progesterone receptor. 



Individual discussion of cases with surgeons having 
positive margins with invasive carcinoma at defini- 
tive surgery has nearly eradicated this problem in 
our hospital. It is clear that a certain feeling of social 
control improves adherence to guidelines. 

Conclusion 

The above review shows that quality control by 
means of internal and external audit and bench- 
marking leads to better breast cancer care. Adher- 
ence to guidelines improves markedly and there are 
data emerging which show that this results in a 
better outcome. As this field of quality control is 
relatively new there is certainly room for more re- 
search on the development of more sophisticated 
tools to measure quality of care. Current QIs are 
sometimes unbalanced, as many indicators are 
encouraging more appropriate care but few indica- 
tors discourage inappropriate care. Particularly in 
older patients life-expectancy calculations may be 
useful to limit unintended harm. If the estimated life 
expectancy and the lag-time-benefit are similar, 
then the net benefits are small or uncertain (Lee et 
al., 2011). A crucial element in quality control is 
the establishment and maintenance of an accurate 
database. This has a considerable economical cost 
which only indirectly benefits the patients. How- 
ever, the advantages are measurable and important. 



Resources will have to be found to pay for man 
(woman) power and computer systems to perform 
quality management. It will be a challenge for the 
medical and hospital community to develop quality 
control systems which are not leading to an exces- 
sive (data input) workflow for clinicians and which 
will not be perceived as an unpleasant "big brother 
is watching you". However, as the speedometer in a 
car is crucial to drive safely, quality measurement is 
of paramount importance to care safely. 
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